1. Field of the Invention
This invention relates to a memory system for the visual display of digital image data and, more particularly, to an embedded ultra-high bandwidth multi-port memory system for ok digital image scaling applications.
2. Description of the Prior Art
Digital image data generally defines one or more frames. A frame is an image displayed for viewing on a display or panel at one time, i.e., one frame of data fits on the display screen or panel. Each frame includes a rectangular array of pixels. Each pixel has one or more values, for example, a gray scale value for a monochrome display or RGB values for a color display. The resolution of the array, i.e., the number of horizontal and vertical pixels, can also be referred to as the image sample rate or resolution. Common display resolutions include that shown in Table 1 indicating, in the second and third columns, the number of pixels in the vertical and horizontal dimensions, respectively:
Where the resolution or sample rate of the display device matches the resolution of the image data, the data can be displayed directly; if not, it is desirable in many cases that the image be appropriately scaled. Scaling can be done in either vertical or horizontal or both dimensions, and the sample rates can be scaled up or down. Increasing the size a digital image (scaling up) is accomplished by introducing additional pixels in either or both the vertical and horizontal directions. The additional pixels can be introduced by linearly interpolating between two existing pixels or by using more sophisticated techniques such as multi-rate Finite Impulse Response (FIR) filters. The use of FIR filters to accomplish vertical and horizontal scaling is described in U.S. Pat. Nos. 4,020,332 to Crochiere, et al., 4,682,301 to Hiroba et al., and 5,355,328 to Arbeiter, et al., all incorporated herein by reference.
FIG. 1 illustrates the relationship of input to output pixels when a FIR filter is used to vertically and horizontally scale an input image. Assume a FIR filter includes 3 multipliers (not shown). In this case, 9 pixels of the input image data contribute to the value of each output pixel. A vertical image scaling circuit (not shown) generates pixel 79 from pixels 70, 71, and 72, pixel 80 from pixels 73, 74, and 75, pixel 81 from pixels 76, 77, and 78, and so on. The vertically scaled image is then provided to a horizontal image scaling circuit (not shown) that generates pixel 82 from pixels 79, 80, and 81. A primary goal of scaling is to maintain the integrity of the image by avoiding distortion due to, e.g., keystoning, warping, or other such effects. Scaling becomes particularly important in connection with pixelated display systemsxe2x80x94devices such as liquid crystal display (LCD) projectors, flat panel monitors, plasma displays (PDP), field emissive displays (FED), electro-luminescent (EL), micro-mirror technology displays (e.g., DMD), etc.xe2x80x94that have a fixed pixel structure.
FIG. 2 is a block diagram of a conventional vertical image scaling circuit 10. A conventional image scaling circuit 10 includes a plurality of line memories such as line memories LM1, LM2, . . . LMi coupled to a vertical scalar 12. Digital image data 11 is input to the line memory LM1 as a stream of pixels representing an image to be scaled and ultimately displayed. To properly scale the digital image data 11 in a vertical dimension, the vertical scalar circuit 12 must have simultaneous parallel access to multiple lines of the digital input data 11 as explained above. Each line memory, e.g., LM1, stores a line of the digital image. Thus, the multiple line memories LM1, LM2, . . . LMi provide the necessary simultaneous parallel access by storing sequential lines of the digital image for a predetermined image frame. The line memories LM1, LM2, . . . LMi are serially connected, that is, line memory LM1 is serially connected to line memory LM2, which is serially connected to line memory LM3, and so on. The vertical image scalar 12 scales the digital image data 11 and provides the scaled data 13 directly to a display device, a horizontal image scalar (not shown), or other circuit block for further processing. Line memories, like line memories LM1, LM2, . . . LMi, are generally large Static Random Access Memory (SRAM) devices capable of storing complete lines of digital image data.
Generally, vertical scalars, like scalar 12, are embedded in Application Specific Integrated Circuits (ASICs) designed for the specific application envisioned, vertically scaling digital images in this case. In addition to line memories, conventional vertical scalar ASICs use external Dynamic Random Access Memory (DRAM) type memory for certain other applications, e.g., frame rate conversion. Thus, vertical scalar ASICs of the type shown in FIG. 1 require both SRAM for the line memories and external DRAM for other applications. The separate SRAM and DRAM requirements increase design complexity, which necessarily increases defect and failure potential. Moreover, the separate large SRAM required for the line memories is often embedded into the vertical scalar ASICs. If embedded, the large SRAM line memory devices use up valuable and costly silicon area.
Accordingly, a need remains for improvements in image scaling methods and apparatus. In particular, a need remains for a simplified image scaling memory system that improves reliability, lowers cost, and improves silicon area usage.
The image scaling memory system of the present invention eliminates the use of internal or external line memories by using an existing frame memory coupled with an input buffer and a plurality of output buffers for providing a vertical scalar with simultaneous parallel access to multiple lines of data. Additionally, the image scaling memory system of the present invention, including the frame memory, is embedded into an integrated circuit. Thus, the image scaling circuit of the present invention improves reliability, lowers cost, and improves silicon area usage.
The frame memory is coupled to an input buffer at an input side and a plurality of output buffers at an output side. The input buffer packs input data pixels to form data words. Once a predetermined number of data words are stored, the input buffer transfers the stored data words to the frame memory. The plurality of output buffers is coupled to the frame memory and the vertical scalar. Each output buffer sequentially gains access to and transfers portions of image lines from the frame buffer. Each output buffer stores only a portion of an image line resulting in relatively small output buffers. The plurality of output buffers provides the vertical scalar with simultaneous parallel access to sequential lines of buffered digital image data. The input and output buffers are preferably implemented using First-In First-Out (FIFO) buffers.
The frame memory preferably comprises DRAM that stores the image data such that row faults are minimized. However, the frame memory may include other suitable memory types, e.g., SRAM. The DRAM frame memory preferably includes at least two memory banks, each including a plurality of rows and a plurality of columns. Each row of the DRAM frame memory contains digital data from multiple lines and stores portions of a plurality of digital data line such that corresponding portions of sequential digital data lines are stored in no more than one row in each bank thereby minimizing row faults. Only one row per bank is active at any given time. There are preferably at least Nxe2x88x921 partial image lines stored in each row of the DRAM frame memory to minimize the row fault rate, where N is the number of lines simultaneously required by the vertical scalar to vertically scale the digital image data. The DRAM frame memory includes an image portion for storing the digital image data frames, an On Screen Display (OSD) portion for storing OSD bitmaps for access by an OSD controller, a microprocessor portion for storing microprocessor data for access by a microprocessor, or a combination thereof.
Another embodiment of the present invention is a vertical scaling circuit embedded in an integrated circuit for vertically scaling digital data of a pixelated image. The vertical scaling circuit comprises a frame memory for storing the digital data, the digital data being divided into a plurality of frames, each frame including a plurality of lines. A plurality of output buffers is coupled to the frame memory. Each output buffer stores at least a portion of a selected line. A vertical scalar is coupled to the plurality of output buffers for vertically scaling the pixelated image by simultaneously parallel accessing the portions of the selected lines stored in each of the output buffers. The vertical scalar can be implemented using a variety of techniques including, but not limited to, the FIR filter techniques described herein.
As mentioned above, the frame memory is preferably DRAM memory but can comprise other similar memory types, e.g., SRAM. Where DRAM is used, the frame memory preferably comprises at least two memory banks, each including a plurality of rows and columns. Each row stores portions of sequential digital image data lines. Row faults are minimized by storing sequential digital data lines in no more than one row in each bank. The DRAM frame memory includes an image portion for storing the digital image frames, an OSD portion for storing OSD bitmaps for access by an OSD controller, a microprocessor portion for storing microprocessor data for access by a microprocessor, or a combination thereof.
The vertical scaling circuit further includes an input buffer coupled to an input side of the frame memory for buffering the digital image data. The input buffer is also embedded in the integrated circuit. The frame memory includes an output port. Each output buffer sequentially accesses the frame memory through the output port and buffers portions of image lines stored therein. The input and plurality of output buffers are preferably implemented using FIFO buffers.
A method for vertically scaling data representative of a pixelated digital image is provided. The method comprises embedding a frame memory and a vertical scalar in an integrated circuit, storing the data in the frame memory, transferring portions of selected data lines from the frame memory to a corresponding output buffer until a plurality of output buffers stores corresponding portions of sequential digital data lines, and simultaneous parallel accessing the portions of selected data lines stored in the plurality of output buffers.
Storing the data in the frame memory includes storing the data in a DRAM memory. Storing the data in the frame memory includes storing the data in at least two memory banks, each bank including a plurality of rows and columns. Each row stores corresponding portions of sequential digital data lines such that corresponding portions of sequential digital data lines are stored in no more than one row in each bank thereby minimizing row faults.
Storing the digital data in the frame memory includes storing a digital data frame in the frame memory for conversion from an input refresh rate to an output refresh rate, storing OSD bitmaps in the frame memory, or storing microprocessor data in the frame memory.
The frame memory includes an output port. Transferring portions of selected digital data lines includes having each output buffer sequentially access the output port. The method further includes buffering the data in an input buffer. The input buffer and the plurality of output buffers are preferably implemented using FIFO buffers.